Facilitating presentation by mobile device of additional content for a word or phrase upon utterance thereof

ABSTRACT

A method for presenting additional content for a word that is part of a message, and that is presented by a mobile communication device, includes the steps of: presenting the message, including emphasizing one or more words for which respective additional content is available for presenting by the mobile communication device; receiving an utterance that includes an emphasized word for which additional content is available for presenting by the mobile communication device; and presenting the additional content for the emphasized word included in the utterance received by the mobile communication device. These steps are performed by the mobile communication device.

I. CROSS-REFERENCE TO RELATED APPLICATION

The present application is a U.S. continuation patent application of,and claims priority under 35 U.S.C. §120 to, U.S. nonprovisional patentapplication Ser. No. 12/198,116, filed Aug. 25, 2008,

-   -   (a) which '116 application is a U.S. continuation-in-part patent        application of, and claims priority under 35 U.S.C. §120 to,        U.S. nonprovisional patent application Ser. No. 12/197,227,        filed Aug. 22, 2008, which '227 application is a nonprovisional        patent application of, and claims priority under 35 U.S.C.        §119(e) to, both U.S. provisional patent application Ser. No.        60/957,386, filed Aug. 22, 2007 and entitled “CONTINUOUS SPEECH        TRANSCRIPTION PERFORMANCE INDICATION”, and U.S. provisional        patent application Ser. No. 60/957,393, filed Aug. 22, 2007 and        entitled “VOICE CLICK FOR SPEECH-ENABLED APPLICATIONS,” and    -   (b) which '116 application is a nonprovisional of, and claims        the benefit under 35 U.S.C. §119(e) to, each of:        -   (1) U.S. provisional patent application Ser. No. 60/957,701,            filed Aug. 23, 2007 and entitled “CONTINUOUS SPEECH            TRANSCRIPTION PERFORMANCE INDICATION;”        -   (2) U.S. provisional patent application Ser. No. 60/957,702,            filed Aug. 23, 2007 and entitled “VOICE CLICK FOR            SPEECH-ENABLED APPLICATIONS;”        -   (3) U.S. provisional patent application Ser. No. 60/972,851,            filed Sep. 17, 2007 and entitled “SYSTEM AND METHOD FOR            DELIVERING MOBILE ADVERTISING WITHIN A THREADED SMS OR IM            CHAT CONVERSATION ON A MOBILE DEVICE CLIENT;”        -   (4) U.S. provisional patent application Ser. No. 60/972,853,            filed Sep. 17, 2007 and entitled “METHOD AND SYSTEM FOR            DYNAMIC PERSONALIZATION AND QUERYING OF USER PROFILES BASED            ON SMS/IM CHAT MESSAGING ON A MOBILE DEVICE;”        -   (5) U.S. provisional patent application Ser. No. 60/972,854,            filed Sep. 17, 2007 and entitled “LOCATION, TIME & SEASON            AWARE MOBILE ADVERTISING DELIVERY;”        -   (6) U.S. provisional patent application Ser. No. 60/972,936,            filed Sep. 17, 2007 and entitled “DELIVERING TARGETED            ADVERTISING TO MOBILE DEVICE FOR PRESENTATION WITHIN SMSes            OR IM CONVERSATIONS;”        -   (7) U.S. provisional patent application Ser. No. 60/972,943,            filed Sep. 17, 2007 and entitled “Dynamic Personalization            and Querying of User Profiles Based on SMSes and IM            Conversations;”        -   (8) U.S. provisional patent application Ser. No. 60/972,944,            filed Sep. 17, 2007 and entitled “Location, Time, and Season            Aware Advertising Delivery to and Presentation on Mobile            Device Within SMSes or IM Conversations or User Interface            Thereof;”        -   (9) U.S. provisional patent application Ser. No. 61/016,586,            filed Dec. 25, 2007 and entitled “VALIDATION OF MOBILE            ADVERTISING FROM DERIVED INFORMATION;”        -   (10) U.S. provisional patent application Ser. No.            61/021,341, filed Jan. 16, 2008 and entitled “CONTINUOUS            SPEECH TRANSCRIPTION UTTERANCE EMPHASIS AND SILENCE            INDICATION;”        -   (11) U.S. provisional patent application Ser. No.            61/034,815, filed Mar. 7, 2008 and entitled “USE OF            INTERMEDIATE SPEECH TRANSCRIPTION RESULTS IN EDITING FINAL            SPEECH TRANSCRIPTION RESULTS;”        -   (12) U.S. provisional patent application Ser. No.            61/038,046, filed Mar. 19, 2008 and entitled “CORRECTIVE            FEEDBACK LOOP FOR AUTOMATED SPEECH RECOGNITION;”        -   (13) U.S. provisional patent application Ser. No.            61/041,219, filed Mar. 31, 2008 and entitled “USE OF            METADATA TO POST PROCESS SPEECH RECOGNITION OUTPUT;” and        -   (14) U.S. provisional patent application Ser. No.            61/091,330, filed Aug. 22, 2008 and entitled “METHODS,            APPARATUSES, AND SYSTEMS FOR PROVIDING TIMELY USER CUES            PERTAINING TO SPEECH RECOGNITION.”

Each of the foregoing patent applications from which priority isclaimed, and any corresponding patent application publications thereof,are hereby incorporated herein by reference in their entirety.Additionally, the disclosure of provisional application 60/789,837 iscontained in Appendix A attached hereto and, likewise, is incorporatedherein in its entirety by reference and is intended to providebackground and technical information with regard to the systems andenvironments of the inventions of the current provisional patentapplication. Similarly, the disclosure of the brochure of Appendix B isincorporated herein in its entirety by reference. Additionally, U.S.provisional patent application Ser. No. 60/957,706, filed Aug. 23, 2007and titled “POST-PROCESSING TRANSCRIPTION RESULTS WITH FILTERS ANDFINITE GRAMMARS,” and U.S. provisional patent application Ser. No.61/021,335, filed Jan. 16, 2008 and titled “USING A PHYSICAL PHENOMENADETECTOR TO START AND STOP RECORDING FOR A SPEECH RECOGNITION ENGINE,”are each incorporated herein by reference.

Finally, the disclosures of each of the following patent applications,and any corresponding patent application publications thereof, areincorporated herein by reference: U.S. nonprovisional patent applicationSer. No. 12/198,112, filed Aug. 25, 2008 and entitled “FILTERINGTRANSCRIPTIONS OF UTTERANCES,” and U.S. nonprovisional patentapplication Ser. No. 12/197,213, filed Aug. 22, 2008 and entitled“CONTINUOUS SPEECH TRANSCRIPTION PERFORMANCE INDICATION.”

II. COPYRIGHT STATEMENT

All of the material in this patent document is subject to copyrightprotection under the copyright laws of the United States and of othercountries. The copyright owner has no objection to the facsimilereproduction by anyone of the patent document or the patent disclosure,as it appears in the governmental files or records, but otherwisereserves all copyright rights whatsoever.

III. BACKGROUND OF THE INVENTION

Mobile applications that display text message threads are often deployedon devices that have limited visual display area, such as mobile phones.Additionally, there may be advertising content linked to elements(words, phrases, punctuation, etc.) of the conversation. Due to thesmall display area, visual advertising content must remain hidden or beas unobtrusive as possible until the user wishes to access it.

One approach is to highlight the elements that have additionaladvertising (or other content) such that no additional space is taken upon the display, but the user is still aware of the visual cue. Textcolor, font, font weight, and the like are all ways that this can beaccomplished.

After the user has been made aware of the visual content cues, they mayselect them to access the additional content. This might be done bynavigating and selecting highlighted cues with a keypad, or by selectingthem directly with a pointing device, such as a mouse, touchpad, or thelike. Unfortunately, this can be unwieldy and inconvenient on a smalldevice, especially when the user is involved in other tasks, such asdriving a car, that require visual attention and which occupy the handsand eyes at the same time.

Thus, a need exists for an approach whereby speech transcription is usedto access additional content associated with textual conversation datawhile minimizing user distraction and the use of hands and eyes. Thisneed is addressed by one or more aspects of the invention.

IV. SUMMARY OF THE INVENTION

The present invention includes many aspects and features. Moreover,while many aspects and features relate to, and are described in, thecontext of instant messaging and SMS messaging, the present invention isnot limited to use only in such contexts, as will become apparent fromthe following summaries and detailed descriptions of aspects, features,and one or more embodiments of the present invention. For instance, theinvention is equally applicable to use in the context of voicemails andemails.

Accordingly, in a first Mobile applications that display text messagethreads are often deployed on devices that have limited visual displayarea, such as mobile phones. Additionally, there may be advertisingcontent linked to elements (words, phrases, punctuation, etc.) of theconversation. Due to the small display area, visual advertising contentmust remain hidden or be as unobtrusive as possible until the userwishes to access it.

One approach is to highlight the elements that have additionaladvertising (or other content) such that no additional space is taken upon the display, but the user is still aware of the visual cue. Textcolor, font, font weight, and the like are all ways that this can beaccomplished.

After the user has been made aware of the visual content cues, they mayselect them to access the additional content. This might be done bynavigating and selecting highlighted cues with a keypad, or by selectingthem directly with a pointing device, such as a mouse, touchpad, or thelike. Unfortunately, this can be unwieldy and inconvenient on a smalldevice, especially when the user is involved in other tasks, such asdriving a car, that require visual attention and which occupy the handsand eyes at the same time.

Thus, a need exists for an approach whereby speech transcription is usedto access additional content associated with textual conversation datawhile minimizing user distraction and the use of hands and eyes. Thisneed is addressed by one or more aspects of the invention.

In a first aspect of the invention, a method for presenting additionalcontent for a word that is part of a message, and that is presented by amobile communication device, includes the steps performed by the mobilecommunication device of: presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device; receiving anutterance that includes an emphasized word for which additional contentis available for presenting by the mobile communication device; andpresenting the additional content for the emphasized word included inthe utterance received by the mobile communication device.

In a feature of the aspect, the mobile communication device is a phone.

In a feature of the aspect, the additional content is presented byplaying audio over a speaker of the mobile communication device.

In a feature of the aspect, said step of presenting the message,including emphasizing one or more words for which respective additionalcontent is available for presenting by the mobile communication device,comprises the step of playing audio of the message on the mobilecommunication device, wherein the audio includes pronunciation emphasisfor the one or more words for which respective additional content isavailable for presenting by the mobile communication device.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofplaying audio of the message on the mobile communication device, whereinthe audio includes aural cueing for the one or more words for whichrespective additional content is available for presenting by the mobilecommunication device.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofdisplaying text of the message on the mobile communication device suchthat the one or more words, for which respective additional content isavailable for presenting on the mobile communication device, arehighlighted. The respective additional content for an emphasized wordmay be presented by expanding the displayed text of the message toinclude the respective additional content in conjunction with theemphasized word.

In a feature, the additional content includes a web address, an emailaddress, or geospatial information.

In a feature, the additional content is presented by displaying a videoon the mobile communication device.

In a feature, the additional content is presented in a web browser ofthe mobile communication device.

In a feature, the additional content is presented in a popup window ofthe mobile communication device.

In a feature, a particular word for which respective additional contentis available for presenting by the mobile communication device consistsof a single word.

In a feature, a particular word for which respective additional contentis available for presenting by the mobile communication device comprisesa phrase.

In a feature, a particular word for which respective additional contentis available for presenting by the mobile communication device comprisesan alphanumeric string.

In a feature, the utterance includes a cue indicating that the utteranceincludes an emphasized word for which respective additional content isavailable for presenting by the mobile communication device.

In a feature, multiple alternative additional content exists for aparticular word that are available for presenting by the mobilecommunication device, and further includes presenting by the mobilecommunication device the additional content of each such alternative.

In another feature, the method further includes the steps of receivingnon-verbal input by the mobile communication device, which inputrepresents a selection of an emphasized word for which respectiveadditional content is to be presented by the mobile communicationdevice; and presenting, by the mobile communication device, theadditional content for the selected word, whereby additional content maybe presented during times when the mobile communication device is unableto successfully receive an utterance.

In a feature, multiple alternative additional content exists for aparticular word that are available for presenting by the mobilecommunication device, and further includes the steps of selecting andpresenting additional content of one of the alternatives.

The step of selecting additional content of one of the alternatives maybe performed by a user of the mobile communication device, or may beperformed in accordance with a computer algorithm.

In another aspect of the invention, a method for presenting additionalcontent for a word that is part of a message and that is presented by amobile communication device, includes the steps performed by the mobilecommunication device of: presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device; communicating anutterance for transcription, the utterance being received by the mobilecommunication device and including an emphasized word that is presentedand for which additional content is available for presenting by themobile communication device; receiving a transcription of the utterance,including a textual representation of the emphasized word; and based onthe textual representation of the emphasized word, presenting theadditional content for such emphasized word.

In a feature, said step of communicating an utterance for transcriptionincludes recording the utterance and sending audio data representing therecorded utterance.

In a feature, said step of communicating an utterance for transcriptionincludes streaming a recording of the utterance.

In a feature, the respective additional content for each of theemphasized words that are presented by the mobile communication deviceis received by and stored in the mobile communication device before thetranscription of the utterance is received by the mobile communicationdevice. The respective additional content in this respect is stored in amemory of the mobile communication device in association with theemphasized word to which the additional content pertains, with theadditional content that is presented being based on a match of thetextual representation of the emphasized word with the emphasized wordsstored in the memory of the mobile communication device.

In a feature, the mobile communication device is a phone.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofplaying audio of the message on the mobile communication device, whereinthe audio includes pronunciation emphasis or aural cueing for the one ormore words for which respective additional content is available forpresenting by the mobile communication device.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofdisplaying text of the message on the mobile communication device suchthat the one or more words, for which respective additional content isavailable for presenting on the mobile communication device, arehighlighted.

In another aspect of the invention, a method for presenting additionalcontent for a word that is part of a message and that is presented by amobile communication device, the steps performed by the mobilecommunication device of: presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device; communicating anutterance for transcription, the utterance being received by the mobilecommunication device and including an emphasized word that is presentedand for which additional content is available for presenting by themobile communication device; in response to communicating the utterancefor transcription, receiving the additional content for the emphasizedword; and presenting the additional content received for the emphasizedword.

In a feature, said step of communicating an utterance for transcriptionincludes recording the utterance and sending audio data representing therecorded utterance.

In a feature, said step of communicating an utterance for transcriptionincludes streaming a recording of the utterance.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofplaying audio of the message on the mobile communication device, whereinthe audio includes pronunciation emphasis or aural cueing for the one ormore words for which respective additional content is available forpresenting by the mobile communication device.

In a feature, said step of presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device, comprises the step ofdisplaying text of the message on the mobile communication device suchthat the one or more words, for which respective additional content isavailable for presenting on the mobile communication device, arehighlighted.

In another aspect of the invention, a method for presenting additionalcontent for a word that is part of a message and that is presented by amobile communication device, includes the steps performed by the mobilecommunication device of: presenting the message, including emphasizingone or more words for which respective additional content is availablefor presenting by the mobile communication device; communicating anutterance for transcription, the utterance being received by the mobilecommunication device and including an emphasized word that is presentedand for which additional content is available for presenting by themobile communication device; in response to communicating the utterancefor transcription, receiving additional content data for the emphasizedword; and presenting the additional content for the emphasized wordbased on the received additional content data.

The additional content data may include a web address, and the step ofpresenting the additional content for the emphasized word based on thereceived additional content data comprises opening the web address in aweb browser of the mobile communication device. The step ofcommunicating an utterance for transcription may include recording theutterance and sending audio data representing the recorded utterance, orthe step of communicating an utterance for transcription may includestreaming a recording of the utterance.

The step of presenting the message, including emphasizing one or morewords for which respective additional content is available forpresenting by the mobile communication device, may include the step ofplaying audio of the message on the mobile communication device, whereinthe audio includes pronunciation emphasis or aural cueing for the one ormore words for which respective additional content is available forpresenting by the mobile communication device, or the step of presentingthe message, including emphasizing one or more words for whichrespective additional content is available for presenting by the mobilecommunication device, may include the step of displaying text of themessage on the mobile communication device such that the one or morewords, for which respective additional content is available forpresenting on the mobile communication device, are highlighted.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of atranscribed utterance intended for a message, includes the steps of,first: receiving audio data communicated from a mobile communicationdevice, the audio data representing an utterance that is intended to beat least a portion of the text of a message that is to be sent from themobile communication device to a recipient; transcribing the utteranceto text based on the audio data; screening the text to identify wordsthat are appropriate for searching; searching for additional content inone or more databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the transcribed text with theidentified words to the mobile communication device such that theidentified words are emphasized when the text is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, the respective additional content foundfor such words during searching, and an identifier of the mobilecommunication device. The method includes the steps of, thereafter,receiving audio data communicated from the mobile communication device,the audio data representing an utterance that includes an identifiedword that is maintained in the database in association with theidentifier of the mobile communication device; transcribing theutterance to text based on the audio data; comparing the transcribedtext to the identified words maintained in the database in associationwith the identifier of the mobile communication device; andcommunicating the respective additional content to the mobilecommunication device that is maintained in the database in associationwith an identified word found in the transcribed text based on thecomparing.

In a feature, said step of communicating to the mobile communicationdevice the respective additional content that is maintained in thedatabase in association with an identified word included in thetranscribed text includes communicating to the mobile communicationdevice the respective additional content that is maintained in thedatabase in association with each identified word that is found in thetranscribed text based on the comparing.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises highlighting thewords such that the words are emphasized when displayed on the mobilecommunication device.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include pronunciationemphasis of the words for which respective additional content is foundin the searching.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include aural cueing for thewords for which respective additional content is found in the searching.

In a feature, the audio data that is received from the mobilecommunication device representing an utterance that is intended to be atleast a portion of the text of a message is received with an indicationof the type of message. The indication may be that the message is aninstant message, or that the message is to be sent from the mobilecommunication device in accordance with a text messaging protocol. Themethod further may include filtering the transcribed text based on theindication.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of atranscribed utterance intended for a message, includes the steps of,first: receiving audio data communicated from a mobile communicationdevice, the audio data representing an utterance that is intended to beat least a portion of the text of a message that is to be sent from themobile communication device to a recipient; transcribing the utteranceto text based on the audio data; screening the text to identify wordsthat are appropriate for searching; searching for additional content inone or more databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the transcribed text with theidentified words to the mobile communication device such that theidentified words are emphasized when the text is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, respective additional content data forsuch words, and an identifier of the mobile communication device. Themethod further includes the steps of, thereafter, receiving audio datacommunicated from the mobile communication device, the audio datarepresenting an utterance that includes an identified word that ismaintained in the database in association with the identifier of themobile communication device; transcribing the utterance to text based onthe audio data; comparing the transcribed text to the identified wordsmaintained in the database in association with the identifier of themobile communication device; and communicating the respective additionalcontent data to the mobile communication device that is maintained inthe database in association with an identified word found in thetranscribed text based on the comparing.

In a feature, the searching includes searching based on an identifiedword using an Internet search engine and analyzing the results providedin response thereto.

In a feature, the additional content data comprises a web address forpassing to a web browser of the mobile communication device.

In a feature, the additional content data comprises a URL for retrievingthe additional content for presentation by the mobile communicationdevice.

In a feature, the searching includes sending a keyword to a third partySMS service and analyzing the results provided in response thereto.

In a feature, the additional content data comprises a keyword forsending to a third party SMS service, in response to which is providedthe additional content found in the searching.

In a feature, said step of receiving audio data communicated from themobile communication device, the audio data representing an utterancethat includes an identified word that is maintained in the database inassociation with the identifier of the mobile communication device,comprises receiving audio data representing an utterance that furtherincludes a cue that an identified word is included in the utterance forpresenting the additional content for such identified word.

In a feature, the association between the identified words, respectiveadditional content data for such words, and an identifier of the mobilecommunication device is maintained in the database for an extendedperiod of time includes a plurality of days or weeks, whereby additionalcontent may be provided for an identified word communicated to themobile communication device even though a plurality of days have passedsince the identified word was communicated to the mobile communicationdevice.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of atranscribed utterance intended for a message includes the steps of,first: receiving audio data communicated from a mobile communicationdevice, the audio data representing an utterance that is intended to beat least a portion of the text of a message that is to be sent from themobile communication device to a recipient; transcribing the utteranceto text based on the audio data; screening the text to identify wordsthat are appropriate for searching; searching for additional content inone or more databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the transcribed text with theidentified words to the mobile communication device such that theidentified words are emphasized when the text is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, respective additional content data forsuch words, and an identifier of the mobile communication device;thereafter, receiving audio data communicated from the mobilecommunication device, the audio data representing an utterance thatincludes an identified word that is maintained in the database inassociation with the identifier of the mobile communication device;transcribing the utterance to text based on the audio data; comparingthe transcribed text to the identified words maintained in the databasein association with the identifier of the mobile communication device;retrieving the respective additional content found in the search byutilizing the additional content data maintained in the database inassociation with an identified word found in the transcribed text basedon the comparing; and communicating the retrieved additional content tothe mobile communication device. The additional content data may be aweb address or a URL for retrieving the addition content, or may be akeyword for sending to a third party SMS service, in response to whichis provided the additional content found in the searching.

In another aspect, a method for presenting additional content for a wordthat is part of a message, and that is presented by a mobilecommunication device, includes the steps of: first, by a server,receiving an identifier of the mobile communication device and audiodata communicated from the mobile communication device, the audio datarepresenting an utterance that is intended to be at least a portion ofthe text of a message that is to be sent from the mobile communicationdevice to a recipient; transcribing the utterance to text based on theaudio data; screening the text to identify words that are appropriatefor searching; searching for additional content in one or more databasesbased on words identified as being appropriate for searching;identifying the words for which respective additional content is foundin the searching; communicating the transcribed text with the identifiedwords to the mobile communication device such that the identified wordsare emphasized when the text is presented by the mobile communicationdevice; and maintaining in a database an association between theidentified words, the respective additional content found for such wordsduring searching, and an identifier of the mobile communication device.The method includes next the step, by the mobile communication device,of presenting the message, including emphasizing the identified wordsfor which the respective additional content is available for presentingby the mobile communication device; and then the steps, by the server,of receiving an identifier of the mobile communication device and audiodata communicated from the mobile communication device, the audio datarepresenting an utterance received by the mobile communication device,the utterance including an identified word that is maintained in thedatabase in association with the identifier of the mobile communicationdevice; transcribing the utterance to text based on the audio data;comparing the transcribed text to the identified words maintained in thedatabase in association with the identifier of the mobile communicationdevice; and communicating the respective additional content to themobile communication device that is maintained in the database inassociation with an identified word found in the transcribed text basedon the comparing. The method then includes the step, by the mobilecommunication device, of presenting the additional content received fromthe server.

In a feature, said step of communicating to the mobile communicationdevice the respective additional content that is maintained in thedatabase in association with an identified word included in thetranscribed text includes communicating to the mobile communicationdevice the respective additional content that is maintained in thedatabase in association with each identified word that is found in thetranscribed text based on the comparing, and the step of presenting theadditional content received from the server by the mobile communicationdevice comprises presenting each respective additional content receivedfrom the server.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises highlighting thewords such that the words are emphasized when displayed on the mobilecommunication device.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include pronunciationemphasis of the words for which respective additional content is foundin the searching.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include aural cueing for thewords for which respective additional content is found in the searching.

In a feature, the audio data that is received from the mobilecommunication device representing an utterance that is intended to be atleast a portion of the text of a message is received with an indicationof the type of message. The indication may indicate that the message isan instant message; or that the message is to be sent from the mobilecommunication device in accordance with a text messaging protocol. Themethod may further include filtering the transcribed text based on theindication.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of atranscribed utterance intended for a message includes the steps of,first, by a server, receiving an identifier of the mobile communicationdevice and audio data communicated from the mobile communication device,the audio data representing an utterance that is intended to be at leasta portion of the text of a message that is to be sent from the mobilecommunication device to a recipient; transcribing the utterance to textbased on the audio data; screening the text to identify words that areappropriate for searching; searching for additional content in one ormore databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the transcribed text with theidentified words to the mobile communication device such that theidentified words are emphasized when the text is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, respective additional content data forsuch words, and an identifier of the mobile communication device. Themethod includes, next, the steps performed by the mobile communicationdevice of presenting the message, including emphasizing the identifiedwords for which the respective additional content is available forpresenting by the mobile communication device. The method then includesthe steps, performed by a server, of receiving an identifier of themobile communication device and audio data communicated from the mobilecommunication device, the audio data representing an utterance receivedby the mobile communication device, the utterance including anidentified word that is maintained in the database in association withthe identifier of the mobile communication device; transcribing theutterance to text based on the audio data; comparing the transcribedtext to the identified words maintained in the database in associationwith the identifier of the mobile communication device; andcommunicating the respective additional content data to the mobilecommunication device that is maintained in the database in associationwith an identified word found in the transcribed text based on thecomparing. The method then includes the steps, performed by the mobilecommunication device, of presenting the additional content based on theadditional content data received from the server.

In a feature, the searching includes searching based on an identifiedword using an Internet search engine and analyzing the results providedin response thereto.

In a feature, the additional content data comprises a web address, andthe step of presenting the additional content based on the receivedadditional content data comprises opening the web address in a webbrowser of the mobile communication device.

In a feature, the additional content data comprises a URL for retrievingthe additional content for presentation by the mobile communicationdevice, and the step of presenting the additional content based on thereceived additional content data comprises passing the URL to anappropriate application of the mobile communication device.

In a feature, the searching includes sending a keyword to a third partySMS service and analyzing the results provided in response thereto.

In a feature, the additional content data comprises a keyword forsending to a third party SMS service, and the step of presenting theadditional content based on the received additional content datacomprises sending the keyword to the third party SMS service andreceiving in response thereto the additional content data.

In a feature, said step of receiving audio data communicated from themobile communication device, the audio data representing an utterancethat includes an identified word that is maintained in the database inassociation with the identifier of the mobile communication device,comprises receiving audio data representing an utterance that furtherincludes a cue that an identified word is included in the utterance forpresenting the additional content for such identified word.

In a feature, the association between the identified words, respectiveadditional content data for such words, and an identifier of the mobilecommunication device is maintained in the database for an extendedperiod of time includes a plurality of days or weeks, whereby additionalcontent may be provided for an identified word communicated to themobile communication device even though a plurality of days have passedsince the identified word was communicated to the mobile communicationdevice.

In a feature, said step of presenting the message, including emphasizingthe identified words for which respective additional content isavailable for presenting by the mobile communication device, comprisesthe step of playing audio of the message on the mobile communicationdevice, wherein the audio includes pronunciation emphasis for theidentified words for which respective additional content is availablefor presenting by the mobile communication device.

In a feature, said step of presenting the message, including emphasizingthe identified words for which respective additional content isavailable for presenting by the mobile communication device, comprisesthe step of playing audio of the message on the mobile communicationdevice, wherein the audio includes aural cueing for the identified wordsfor which respective additional content is available for presenting bythe mobile communication device.

In a feature, said step of presenting the message, including emphasizingthe identified words for which respective additional content isavailable for presenting by the mobile communication device, comprisesthe step of displaying text of the message on the mobile communicationdevice such that the identified words, for which respective additionalcontent is available for presenting on the mobile communication device,are highlighted.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of the textof a message presented by a mobile communication device includes thesteps of, first: screening the text of a message to identify words thatare appropriate for searching; searching for additional content in oneor more databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the identified words to themobile communication device such that the identified words areemphasized when the text of the message is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, the respective additional content foundfor such words during searching, and an identifier of the mobilecommunication device to which the identified words were communicated.The method further includes the steps of, thereafter, receiving from themobile communication device to which the identified words werecommunicated the identifier of the mobile communication device and audiodata representing an utterance that includes an identified word that ismaintained in the database in association with the identifier of themobile communication device; transcribing the utterance to text based onthe audio data; comparing the transcribed text to the identified wordsmaintained in the database in association with the identifier of themobile communication device; and communicating the respective additionalcontent to the mobile communication device that is maintained in thedatabase in association with an identified word found in the transcribedtext based on the comparing.

In a feature, the mobile communication device is a mobile phone.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises highlighting thewords such that the words are emphasized when displayed on the mobilecommunication device.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include pronunciationemphasis of the words for which respective additional content is foundin the searching.

In a feature, the step of identifying the words for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the transcription to include aural cueing for thewords for which respective additional content is found in the searching.

In a feature, the message is an instant message and the mobilecommunication device is the recipient of the instant message.

In a feature, the message is an instant message and the mobilecommunication device is the sender of the instant message.

In a feature, the message is a text message and the mobile communicationdevice is the recipient of the text message.

In a feature, the message is a text message and the mobile communicationdevice is the sender of the text message.

In another aspect of the invention, a method of facilitating thepresentation of additional content for a word that is part of the textof a message presented by a mobile communication device includes thesteps of, first: screening the text of a message to identify words thatare appropriate for searching; searching for additional content in oneor more databases based on words identified as being appropriate forsearching; identifying the words for which respective additional contentis found in the searching; communicating the identified words to themobile communication device such that the identified words areemphasized when the text of the message is presented by the mobilecommunication device; and maintaining in a database an associationbetween the identified words, respective additional content data forsuch words, and an identifier of the mobile communication device towhich the identified words were communicated. The method includes thesteps of, thereafter: receiving from the mobile communication device towhich the identified words were communicated the identifier of themobile communication device and audio data representing an utterancethat includes an identified word that is maintained in the database inassociation with the identifier of the mobile communication device;transcribing the utterance to text based on the audio data; comparingthe transcribed text to the identified words maintained in the databasein association with the identifier of the mobile communication device;and communicating the respective additional content data to the mobilecommunication device that is maintained in the database in associationwith an identified word found in the transcribed text based on thecomparing.

In a feature, the searching includes searching based on an identifiedword using an Internet search engine and analyzing the results providedin response thereto.

In a feature, the additional content data comprises a web address forpassing to a web browser of the mobile communication device.

In a feature, the additional content data comprises a URL for retrievingthe additional content for presentation by the mobile communicationdevice.

In a feature, the searching includes sending a keyword to a third partySMS service and analyzing the results provided in response thereto.

In a feature, the additional content data comprises a keyword forsending to a third party SMS service, in response to which is providedthe additional content found in the searching.

In a feature, said step of receiving audio data communicated from themobile communication device, the audio data representing an utterancethat includes an identified word that is maintained in the database inassociation with the identifier of the mobile communication device,comprises receiving audio data representing an utterance that furtherincludes a cue that an identified word is included in the utterance forpresenting the additional content for such identified word.

In a feature, the association between the identified words, respectiveadditional content data for such words, and an identifier of the mobilecommunication device is maintained in the database for an extendedperiod of time includes a plurality of days or weeks, whereby additionalcontent may be provided for an identified word communicated to themobile communication device even though a plurality of days have passedsince the identified word was communicated to the mobile communicationdevice.

In a feature, the steps of the method are performed by a text messagingservice provider.

In a feature, the steps of the method are performed by an InstantMessaging service provider.

In a feature, said step of communicating the identified words to themobile communication device such that the identified words areemphasized when the text of the message is presented by the mobilecommunication device includes communicating the text of the message withthe identified words to the mobile communication device.

In addition to the aforementioned aspects and features of the presentinvention, it should be noted that the present invention furtherencompasses the various possible combinations and subcombinations ofsuch aspects and features.

V. BRIEF DESCRIPTION OF THE DRAWINGS

Further aspects, features, embodiments, and advantages of the presentinvention will become apparent from the following detailed descriptionwith reference to the drawings, wherein:

FIG. 1 is a diagram of a communication system in which one or morepreferred embodiments of the invention may be practiced;

FIG. 2 is a diagram illustrating two users having respectively mobilephones that can be used to communicate back and forth using a portion ofthe communication system of FIG. 1;

FIG. 3 is a diagram of the portion of the communication system of FIG. 1over which the users communicate;

FIG. 4 is a transcription of a chronological sequence of communications,between the two users, that transpires using the mobile phones and theportion of the communication system shown in FIG. 3;

FIG. 5 is a graphical illustration of the chronological sequence ofcommunications of FIG. 4 as it might appear on a display of an instantmessaging (IM) client on one of the users' mobile phones;

FIG. 6 is a diagram illustrating the second user utilizing a “voiceclick” feature in accordance with a preferred embodiment of theinvention;

FIG. 7 is a is graphical illustration of the result of the second userutilizing the “voice click” feature as it might appear on the user'smobile phone;

FIG. 8A is a graphical illustration showing the sequence of messages ina preferred implementation of the invention, wherein content searchingand keyword highlighting are performed by the ATFS in the portion of thecommunication system of FIG. 3;

FIG. 8B is a diagram of the portion of the communication system of FIG.3 over which the sequence of messages are communicated in FIG. 8A

FIG. 9 is a flowchart of steps in a method performed by the ATFS inaccordance with the preferred implementation of FIG. 8A;

FIG. 10 is a graphical illustration showing the sequence of textmessages in another preferred implementation of the invention, whereincontent searching and keyword highlighting are performed by the ATFS andthe mobile communication service provider in the portion of thecommunication system of FIG. 3;

FIG. 11 is a flowchart of steps in a method performed by the mobilecommunication service provider in accordance with the preferredimplementation of FIG. 10;

FIG. 12 is a graphical illustration showing the sequence of textmessages in yet another preferred implementation of the invention,wherein content searching and keyword highlighting are performed by boththe ATFS and the mobile communication service provider in the portion ofthe communication system of FIG. 3;

FIG. 13 is a graphical illustration showing the sequence of messages ina preferred implementation of the invention, wherein content searchingand keyword highlighting are performed by the ATFS 18 in the portion ofthe communication system of FIG. 3;

FIG. 14 is a flowchart of steps in a method performed by the ATFS inaccordance with the preferred implementation of FIG. 13;

FIG. 15 is a graphical illustration showing the sequence of textmessages in another preferred implementation of the invention, whereincontent searching and keyword highlighting are performed by the IMservice provider in the portion of the communication system of FIG. 3;

FIG. 16 is a flowchart of steps in a method performed by the IM serviceprovider in accordance with the preferred implementation of FIG. 15; and

FIG. 17 is a graphical illustration showing the sequence of textmessages in yet another preferred implementation of the invention,wherein content searching and keyword highlighting are performed by boththe ATFS and the IM service provider in the portion of the communicationsystem of FIG. 3.

VI. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

As a preliminary matter, it will readily be understood by one havingordinary skill in the relevant art (“Ordinary Artisan”) that the presentinvention has broad utility and application. Furthermore, any embodimentdiscussed and identified as being “preferred” is considered to be partof a best mode contemplated for carrying out the present invention.Other embodiments also may be discussed for additional illustrativepurposes in providing a full and enabling disclosure of the presentinvention. Moreover, many embodiments, such as adaptations, variations,modifications, and equivalent arrangements, will be implicitly disclosedby the embodiments described herein and fall within the scope of thepresent invention.

Accordingly, while the present invention is described herein in detailin relation to one or more embodiments, it is to be understood that thisdisclosure is illustrative and exemplary of the present invention, andis made merely for the purposes of providing a full and enablingdisclosure of the present invention. The detailed disclosure herein ofone or more embodiments is not intended, nor is to be construed, tolimit the scope of patent protection afforded the present invention,which scope is to be defined by the claims and the equivalents thereof.It is not intended that the scope of patent protection afforded thepresent invention be defined by reading into any claim a limitationfound herein that does not explicitly appear in the claim itself.

Thus, for example, any sequence(s) and/or temporal order of steps ofvarious processes or methods that are described herein are illustrativeand not restrictive. Accordingly, it should be understood that, althoughsteps of various processes or methods may be shown and described asbeing in a sequence or temporal order, the steps of any such processesor methods are not limited to being carried out in any particularsequence or order, absent an indication otherwise. Indeed, the steps insuch processes or methods generally may be carried out in variousdifferent sequences and orders while still falling within the scope ofthe present invention. Accordingly, it is intended that the scope ofpatent protection afforded the present invention is to be defined by theappended claims rather than the description set forth herein.

Additionally, it is important to note that each term used herein refersto that which the Ordinary Artisan would understand such term to meanbased on the contextual use of such term herein. To the extent that themeaning of a term used herein—as understood by the Ordinary Artisanbased on the contextual use of such term—differs in any way from anyparticular dictionary definition of such term, it is intended that themeaning of the term as understood by the Ordinary Artisan shouldprevail.

Furthermore, it is important to note that, as used herein, “a” and “an”each generally denotes “at least one,” but does not exclude a pluralityunless the contextual use dictates otherwise. Thus, reference to “apicnic basket having an apple” describes “a picnic basket having atleast one apple” as well as “a picnic basket having apples.” Incontrast, reference to “a picnic basket having a single apple” describes“a picnic basket having only one apple.”

When used herein to join a list of items, “or” denotes “at least one ofthe items,” but does not exclude a plurality of items of the list. Thus,reference to “a picnic basket having cheese or crackers” describes “apicnic basket having cheese without crackers”, “a picnic basket havingcrackers without cheese”, and “a picnic basket having both cheese andcrackers.” Finally, when used herein to join a list of items, “and”denotes “all of the items of the list.” Thus, reference to “a picnicbasket having cheese and crackers” describes “a picnic basket havingcheese, wherein the picnic basket further has crackers,” as well asdescribes “a picnic basket having crackers, wherein the picnic basketfurther has cheese.”

Referring now to the drawings, in which like numerals represent likecomponents throughout the several views, the preferred embodiments ofthe present invention are next described. The following description ofthe preferred embodiment(s) is merely exemplary in nature and is in noway intended to limit the invention, its application, or uses.

FIG. 1 is a diagram of a communication system 10 in which one or morepreferred embodiments of the invention may be practiced. As showntherein, the communication system 10 includes at least one transmittingdevice 12 and at least one receiving device 14, one or more networksystems 16 for connecting the transmitting device 12 to the receivingdevice 14, and an automatic transcription & filtering (“ATFS”) system18, including a voice-to-text or automatic speech recognition (“ASR”)engine. Transmitting and receiving devices 12,14 may include mobilephones 21, smart phones 22, PDAs 23, tablet notebooks 24, variousdesktop and laptop computers 25,26,27, two-way pagers, and the like. Oneor more of the devices 12,14, such as the illustrated iMac and laptopcomputers 25,26, may connect to the network systems 16 via wirelessaccess points 28. The various transmitting and receiving devices 12,14(one or both types of which being sometimes referred to herein as“client devices”) may be of any conventional design and manufacture.

It will be appreciated that FIG. 1 is intended primarily to providecontext in which inventive aspects and features of the present inventionmay be practiced. Furthermore, in the context of text messaging, thecommunication system 10 preferably includes, inter alia, atelecommunications network. In the context of instant messaging, thecommunications systems 10,60 each preferably includes, inter alia, theInternet and an Instant Messaging (IM) service provider and associatedserver.

FIG. 2 is a diagram illustrating two users 32,34 having respectivemobile phones 12,14 that can be used to communicate back and forth usinga portion of the communication system of FIG. 1. A more detailed view ofthe portion of the communication system 10 is illustrated in FIG. 3,which is a diagram of the portion of the communication system of FIG. 1over which the users 32,34 communicate. As shown therein, a first user32, sometimes referred to herein as Adam, communicates with a seconduser 34, sometimes referred to herein as Betty, by way of respectivehandheld communication devices 12,14. More particularly, Adam and Bettyexchange text messages using their mobile phones, each phone having amicrophone, a speaker, and a display. In the context of text messaging,mobile phones include text messaging applications when sold at retail,and carriers typically charge a small fee for each text message that issent. The text messages typically are sent using the short messageservice (SMS) protocol.

The text messages that are sent preferably are generated by speakinginto the mobile phones, with the utterances being converted to text bythe ATFS 18 in accordance with the disclosure of U.S. Patent Appl. Pub.No. US 2007/0239837. In such disclosure, the ATFS 18 is implemented inone or more backend servers 160, and the one or more network systems 16include transceiver towers 130; one or more mobile communication serviceproviders 140; and the Internet 150.

In operation, one or more transceiver towers 130A receive from themobile phone 12 a message that includes audio data representing arecorded utterance (A). The message (A) is transmitted from the towers130A to a mobile communication service provider 140 and then over theInternet 150 to the ATFS 18 and, specifically, to a backend server 160that performs the transcription of the recorded utterance. Thetranscription may be filtered in accordance with the disclosure of U.S.Patent Appl. Pub. No. US 2007/0239837 so as to include abbreviations andother jargon typically used in text messaging. Once transcribed andfiltered, a message that includes the recorded utterance, now convertedto text (T), is sent back to the mobile phone 12. In particular, themessage (T) is sent over the Internet 150, to the mobile communicationservice provider 140, over the towers 130A, and to the mobile device 12.

Once received by the mobile phone 12, the transcribed and filtered textis reviewable by Adam for proofing and correction. Once verified, thetext is sent in a text message (M) to Betty by sending the message (T)over the towers 130A, to the mobile communication service provider 140,then over the towers 130B and, ultimately, to the mobile phone 14.Furthermore, it should be noted that in this and the following examples,it is assumed that Adam and Betty utilize the same mobile communicationservice provider 140; in the alternative, it will be appreciated that asecond mobile communication service provider could be inserted in thispath after the first mobile phone service provider.

The messages may be displayed individually or as part of a textconversation. FIG. 4 is a transcription of a chronological sequence ofcommunications, between the two users 12,14, that transpires using themobile phones 12,14 and the portion of the communication system 10 shownin FIG. 3, and FIG. 5 is a graphical illustration of this chronologicalsequence of communications of FIG. 4 as it might appear on a display ofan instant messaging (IM) client on one of the users' mobile phones12,14. In accordance with the present invention, certain words arehighlighted in the display when the text of communications are displayedin order to draw attention to those words relative to other displayedwords. The words that are highlighted represent words for whichadditional content may be selected for presentation to a user, such asby display or otherwise, the additional content relating in some mannerto the highlighted word. Moreover, the additional content presented maybe audio (and thus played); visual (and thus displayed); or audiovisual(and thus the audio portion played and the visual portion displayed).

Thus, for example, as shown in the graphical illustration of FIG. 4, thewords “sushi” and “movies” are “highlighted” by being shown underlinedand in bold on the phone's display. As used herein with reference topreferred embodiments, “word” (sometimes “keyword”) generally means asingle word or a phrase, i.e., a collection of words, and may be analphanumeric string, including alphanumeric strings commonly used intext messaging and instant messaging (sometimes “textspeak”). Moreover,as used herein with reference to preferred embodiments, “highlighted”(sometimes “activated”) generally means a word having an attribute thatis different from most other words with which it is displayed forpurposes of drawing a person's attention to that word when displayedwith the other words. Highlighting is setting the attribute of such wordso that it is highlighted. Of course, other forms of highlighting,including varying font size and/or color, background highlighting,animated effects (e.g., blinking) or the like may likewise be utilizedsingularly or in combination.

In further accordance with preferred embodiments of the invention, andas further described in greater detail hereinbelow, the highlightedwords are words for which additional content has been identified fordisplay or other presentation to the user, such as by display orotherwise, and highlighted words may be selected by a user, with theselection of a highlighted word resulting in the display of therespective additional content for the highlighted word. For example,FIG. 6 illustrates Betty speaking “sushi” into her mobile phone 14 forpurposes of selecting the highlighted word “sushi.” In response, apop-up, browser window or the like opens on the display of the mobilephone 14. For example, FIG. 7 is a graphical illustration of a pop-up,displayed in response to Betty's utterance, that provides informationabout “Suzy's Sushi Shack” restaurant. In the illustration, theinformation includes the address, operating hours, and phone number, butany information or other content data may alternatively or additionallybe provided. In a further feature, the information may be tailoredbased, in part, on the location of Betty's mobile phone 14, whereby thelocation of the information (such as a particular nearby restaurant)will be close in proximity to Betty's location. This may be facilitatedwhen Betty's phone includes GPS capabilities, and Betty's location iscommunicated by her mobile phone, either with an utterance or otherwise.

The highlighting and provision of the additional content for display canbe accomplished through various implementations of preferred embodimentsof the invention. Accordingly, six different implementations are nowdescribed, in any of which the foregoing exemplary scenario is enabled.

In each case, one or more system elements are equipped to carry out oneor more functions pertaining to the process of identifying words in thetext message for which content is available and/or of determining thecircumstances under which the content is to be made available.Sub-functions that may be utilized include, but are not limited to:screening the message to identify the set of words that are to beconsidered for activation/highlighting; determining the content sourcesfrom which possible content is to be sought; and using additional data,in combination or association with the words themselves, to moreprecisely control the content/content data to be associated with eachactivated/highlighted word. Furthermore, user preferences, some of whichare described herein, may be incorporated into one or more of thesefunctions. Further details of these various functions are providedbelow.

For each message, there may be one or more words for which content datais to be sought. Although it is possible for every word in the messageto be considered for content association, it may be preferable for oneor more of the words to be removed from consideration, thereby improvingperformance by reducing the number of words and by reducing the amountof irrelevant content data that is ultimately returned to the user.These words may be identified in one or more ways. For example, certainvery common words, including prepositions and articles, are preferablyremoved from consideration (except perhaps if included within a longerphrase), may be removed as being too common for any meaningful contentto be associated therewith. Other more sophisticated techniques maylikewise be applied to reduce the words under consideration. In a simpleexample, the only words considered for further content association maybe the terms which are capitalized. Still further, combinations of words(phrases) may be considered, or only single words may be considered, andin each case, textspeak may or may not be considered. In at least oneembodiment, these various possibilities are established only by thesystem owner or implementer and may not be varied by the user. In atleast one other embodiment, however, one or more of these parameters areuser preferences that pertain only to a specific user and that may beset and adjusted by that user. For example, each user could control, viapreference settings, whether capitalized terms or all terms areconsidered for additional content, or whether single words, phrases of acertain length, or textspeak are considered for additional content. Userpreferences may be established directly via the client application inthe device 12,14, indirectly via a web application, or both.

In seeking content data to be associated with the words identified forconsideration according to one or more of the techniques describedabove, the system may utilize a variety of data sources. In at least oneembodiment, a search is conducted of each selected word (including eachidentified single word, phrase, or textspeak word) using a generalsearch of the Internet using a search engine such as that provided byGoogle, Yahoo!, or Microsoft. In at least one other embodiment, such asearch is conducted using only particular databases. For example, asearch may be conducted in a movie database, local directory, sportsdatabase, or the like for each word that is selected as described above.In another example, the searching includes sending a keyword to a thirdparty SMS service and analyzing the results provided in responsethereto; in this regard, assuming additional content is found, theadditional content data comprises a keyword for thereafter sending to athird party SMS service, in response to which is provided the additionalcontent. A well known and free SMS service is provided by Google,further information about which can be found athttp://www.google.com/intl/en_us/mobile/default/sms/index.html.

In addition to the type of conventional searching described above, in atleast one embodiment, a database of particular words is maintained bythe service provider. More particularly, particular words of interestare listed in a database together with any content data which aninterested party has an interest in being associated with the particularword. For example, a movie theater chain, a movie theater chain, aparticular movie theater, a film studio, a movie distributor or a videostore may have an interest in particular information relating to theirrespective business being associated with every occurrence of the word“movie” in a message, such that that information is provided as contentdata when selected by a user. In the illustrated example, Suzy's SushiShack may have an interest in brief information about a specificrestaurant being associated with the word “sushi,” such that selectionof the word “sushi” as shown in FIG. 6 causes information to bepresented to the user as shown in FIG. 7.

User preferences may once again be utilized in this regard. For example,a user may be able to select types of information, for example from alist, that the user might be interested in. Such categories may includebut are not limited to “movies,” “dining,” “retail stores,” and thelike. In such an arrangement, the word “sushi” might not be activated orhighlighted if the user does not have “dining” or any other categoryselected for association with additional content.

Additional data may also be combined with any of the foregoing. Forexample, the location of one or both devices 12,14, available via GPSenablement and the like, may be taken into consideration. Thus, a movietheater chain may have an interest in having information about the movietheater, in the chain, that is geographically closest to the device12,14 being associated with the word “movie,” and Suzy's Sushi Shack,which may only have one location, may have an interest in havinginformation about its single restaurant being associated with the word“sushi” only if the device is within a predetermined distance from thatlocation. User preferences may be utilized here as well, for example tocontrol the distance (radius) within which the user has an interest inassociated information.

Other searches that may be performed include a search, for a particularmovie, of movie listings for a theater that is nearby the location ofthe mobile phone; a search, for a particular business name or for aparticular category, of business listings that are proximate thelocation of the mobile phone; a search of song titles for a particularsong; a search of book titled for a particular book; a search of stockticker symbols for a particular stock ticker symbol; a search of sportsteams and scores for a particular sports team; a search of a map for animage or directions for a particular address or zip code; and a searchof flight information for a particular flight.

In accordance with a first preferred embodiment, the two users 12,14(generally referred to hereinafter only as “Adam” and “Betty” forconvenience) communicate using text messaging with their mobile phones12,14. Furthermore, Adam uses the ATFS 18 so that he may text Bettywithout typing in the text of his text messages. Such an embodiment isnext discussed with reference to FIGS. 8A, 8B, and 9, wherein FIG. 8A isa graphical illustration showing the sequence of messages in a preferredimplementation of the invention, wherein content searching and keywordhighlighting are performed by the ATFS 18 in the portion of thecommunication system 10 of FIG. 3; FIG. 8B is a diagram of the portionof the communication system of FIG. 3 over which the sequence ofmessages are communicated in FIG. 8A; and FIG. 9 is a flowchart of stepsin a method performed by the ATFS 18 in accordance with the preferredimplementation of FIG. 8A.

As shown therein, Adam utters a message to be sent to Betty. Theutterance is recorded in the form of audio data (A) and is communicated801 over networks 16 in system 10 to a mobile communication serviceprovider 140 and then communicated 803 via the Internet 150 to the ATFS18, which is implemented by one or more servers 160 (collectively andsingly referred to herein as a “server”). In accordance with thisimplementation, the ATFS 18 receives and transcribes 901 the recordedutterance from Adam and screens 903 the transcription for words that areappropriate for highlighting according to one or more of the techniquesdescribed previously (e.g., words like “the” and “a” are preferably notconsidered appropriate for highlighting; phrases may also be identifiedduring screening, as opposed to consideration of just single words; userpreferences may be taken into account, etc.).

For each word found in the screening 903, the ATFS 18 searches 905 foradditional content. Furthermore, the search preferably results in only“meaningful” additional content based on intelligent algorithms andrelevance scoring. Those words for which additional content is found inthe search are then highlighted 907 by the ATFS 18, and the transcribedand filtered text (T)—including the highlighted keywords—is communicated805 to the mobile communication service provider 140 and thencommunicated 807 to Adam's mobile phone 12. Additionally, the ATFS 18maintains 911 an association of the highlighted words, additionalcontent data for the respective additional content identified in thesearch for each of the respective screened words, and an identifier ofthe recipient (e.g., Adam's phone 12) to which the highlighted wordswere sent 909.

Thereafter, when a subsequent recorded utterance (U) from Adam's phone12 is communicated 809,811 and received and transcribed to text 913, thetranscription is compared to the highlighted words that are maintainedin association with the identifier for Adam's phone 12. Upon a match, itis determined that one of the highlighted words has been selected byAdam for display of the additional content on Adam's phone.Consequently, the additional content data (C) maintained in associationwith the highlighted word of the match is sent 917 to Adam's mobilephone 12 for display of the additional content on Adam's mobile phone12. The additional content data (C) may comprise the additional contentfor display or, preferably, the additional content data comprises anInternet web address. The additional content data is communicated813,815 to Adam's mobile phone 12, whereby, upon receipt, the mobilephone 12 is controlled to present content to the user. In this regard,it will be appreciated that if the content data includes a web address(e.g., www.suzysushi.com), such address may be passed to the web browserapplication of the mobile phone 12 for display of the web page via theweb browser. As shown in the message sequence of FIG. 8A, Adam thensends 817,819 a text message to Betty, and Betty may respond 821,823with a text message of her own.

As used herein with reference to preferred embodiments, “additionalcontent” is intended to mean information that is associated with ahighlighted word and that is not necessarily displayed when thehighlighted word is displayed, but that is displayed when thehighlighted word is selected for display. In contrast, as used hereinwith reference to preferred embodiments, “additional content data”generally means a pointer, such as a URL, whereat additional content isavailable for display, but also may mean the additional content itself.Furthermore, as used herein with reference to preferred embodiments,“voice click” is intended to mean the selection of a highlighted word byvoice for the intended display of additional information associated withthe selected word. Furthermore, it will be appreciated from FIG. 8A thatAdam utilizes the “voice click” feature of the invention in thisillustration and that Betty does not utilize the “voice click” featurein this illustration.

Another implementation in accordance with a preferred embodiment of theinvention is described with reference now to FIGS. 10-11, wherein FIG.10 is a graphical illustration showing the sequence of text messages insuch preferred implementation of the invention, wherein contentsearching and keyword highlighting are performed by the ATFS 18 and themobile communication service provider 140 in the portion of thecommunication system 10 of FIG. 3; FIG. 11 is a flowchart of steps in amethod performed by the mobile communication service provider 140 inaccordance with the preferred implementation of FIG. 10. Similar to thecommunication of FIGS. 8A, 8B and 9, Adam and Betty communicate usingtext messaging with their mobile phones 12,14. Furthermore, Adam usesthe ATFS 18 so that he may text Betty without typing in the text of histext messages.

Specifically, as illustrated Adam first utters a message to be sent toBetty. The utterance is recorded in the form of audio data and iscommunicated 1001 to mobile communication service provider 140 and thencommunicated 1003 to the ATFS 18. The utterance is transcribed to textby the ATFS 18 and then communicated 1005,1007 back to Adam's phone 12.This part of the sequence is in accordance with the disclosure of U.S.Patent Appl. Publ. No. 2007/0239837.

Adam next sends 1009 the transcribed text in a text message. Inaccordance with this implementation, the mobile communication serviceprovider 140 receives 1101 and screens 1103 the text message forappropriate words for searching. For each appropriate word found in thescreening 1103, the mobile communication service provider 140 searches1105 for additional content.

Furthermore, the search preferably results in only “meaningful”additional content based on intelligent algorithms and relevancescoring. Those words for which additional content is found in the searchare then highlighted 1107 by the mobile communication service provider140. The text message, including the highlighted keywords, is thencommunicated 1011 to Betty's mobile phone 14. Additionally, the mobilecommunication service provider 140 maintains 1111 an association of thehighlighted words, additional content data for the respective additionalcontent identified in the search for each of the respective wordsidentified in the screening, and an identifier of the recipient (e.g.,Betty's phone 14) to which the highlighted words were sent 1109.

Thereafter, a subsequent recorded utterance from Betty's phone 14 may becommunicated 1013 and received 1113 and transcribed 1115 to text by themobile communication service provider 140, whereupon the transcriptionis compared 1115 to the highlighted words that are maintained inassociation with the identifier for Betty's phone 14. Upon a match, itis determined that one of the highlighted words has been selected byBetty for display of the additional content on Betty's phone 14.Consequently, the additional content data maintained in association withthe highlighted word of the match is sent 1117 to Betty's mobile phone14 for display of the additional content on Betty's mobile phone 14. Theadditional content data (C) may comprise the additional content fordisplay or, preferably, the additional content data comprises anInternet web address. The additional content data is communicated toBetty's mobile phone 14, whereby, upon receipt, the mobile phone 14 iscontrolled to present content to the user. In this regard, it will beappreciated that if the content data includes a web address (e.g.,www.suzysushi.com), such address may be passed to the web browserapplication of the mobile phone 14 for display of the web page via theweb browser.

It will be appreciated that in this implementation, the mobilecommunication service provider 140 may utilize the same technology asthe ATFS 18 (i.e., the mobile communication service provider 140 mayitself include an ASR engine and other components found in the ATFS 18)in receiving, transcribing, and comparing an utterance to highlightedwords taken from messages for which words additional content has beenidentified. Moreover, the ability of the mobile communication serviceprovider 140 to do this enables highlighting of words in text messagesfrom Betty to Adam even when Betty manually types the text messages onher mobile phone 14.

For example, as shown in the message sequence of FIG. 10, Betty sends1017 a text message to Adam. The mobile communication service provider140 receives and screens the text message from Betty for appropriatewords for searching; for each appropriate word, the mobile communicationservice provider 140 searches for additional content; and those wordsfor which additional content is found in the search are thenhighlighted. The text message—including the highlighted keywords—is thencommunicated 1019 to Adam's mobile phone 12. Additionally, the mobilecommunication service provider 140 maintains an association (preferablyin a database) of the highlighted words, additional content data for therespective additional content identified in the search for each of therespective words identified in the screening, and an identifier ofAdam's phone 12 to which the highlighted words were communicated 1019.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1021 and received and transcribed to text by the mobilecommunication service provider 140, whereupon the transcription iscompared to the highlighted words that are maintained in associationwith the identifier for Adam's phone 12. Upon a match, it is determinedthat one of the highlighted words has been selected by Adam for displayof the additional content on Adam's phone 12. Consequently, theadditional content data maintained in association with the highlightedword of the match is sent 1023 to Adam's mobile phone 12 for display ofthe additional content on Adam's mobile phone 12.

A third implementation in which text messaging between Adam and Betty isillustrated is shown in FIG. 12, which simply represents the combinationof the first and second implementations, i.e., in this thirdimplementation, both the ATFS 18 and the mobile communication serviceprovider 140 separately and independently support voice clicking Moreparticularly, FIG. 12 is a graphical illustration showing the sequenceof text messages in such an implementation of the invention, whereincontent searching and keyword highlighting are performed by both theATFS 18 and the mobile communication service provider 140 in the portionof the communication system 10 of FIG. 3.

Specifically, Adam utters a message to be sent to Betty. The utteranceis recorded in the form of audio data and is communicated 1201 to mobilecommunication service provider 140 and then communicated 1203 to theATFS 18. In accordance with this implementation, the ATFS 18 receivesand transcribes the recorded utterance from Adam and screens thetranscription for words that are appropriate for highlighting.

For each word found in the screening, the ATFS 18 searches foradditional content. Furthermore, the search preferably results in only“meaningful” additional content based on intelligent algorithms andrelevance scoring. Those words for which additional content is found inthe search are then highlighted by the ATFS 18, and the transcribed andfiltered text—including the highlighted keywords—are communicated 1205to the mobile communication service provider 140 and then communicated1207 to Adam's mobile phone 12. Additionally, the ATFS 18 maintains anassociation (preferably in a database) of the highlighted words,additional content data for the respective additional content identifiedin the search for each of the respective screened words, and anidentifier of the recipient (e.g., Adam's phone 12) to which thehighlighted words were sent.

Thereafter, when a subsequent recorded utterance from Adam's phone 12 iscommunicated 1209,1211 and received by the ATFS 18 and transcribed totext, the transcription is compared to the highlighted words that aremaintained in association with the identifier for Adam's phone 12. Upona match, it is determined that one of the highlighted words has beenselected by Adam for display of the additional content on Adam's phone.Consequently, the additional content data maintained in association withthe highlighted word of the match is communicated 1213,1215 to Adam'smobile phone 12 for display of the additional content on Adam's mobilephone 12. The additional content data (C) may comprise the additionalcontent for display or, preferably, the additional content datacomprises an Internet web address. The additional content data iscommunicated to Adam's mobile phone 12, whereby, upon receipt, themobile phone 12 is controlled to present content to the user. In thisregard, it will be appreciated that if the content data includes a webaddress (e.g., www.suzysushi.com), such address may be passed to the webbrowser application of the mobile phone 12 for display of the web pagevia the web browser.

As shown in the message sequence of FIG. 12, Adam then sends 1217 a textmessage to Betty that includes the transcribed and filtered textreceived from the ATFS 18, but that does not include the highlightedwords. In accordance with this implementation, the mobile communicationservice provider 140 receives and screens the text message forappropriate words for searching. For each appropriate word found in thescreening, the mobile communication service provider 140 searches foradditional content.

Furthermore, the search preferably results in only “meaningful”additional content based on intelligent algorithms and relevancescoring. Those words for which additional content is found in the searchare then highlighted by the mobile communication service provider 140.The text message—including the highlighted keywords—is then communicated1219 to Betty's mobile phone 14. Additionally, the mobile communicationservice provider 140 maintains an association of the highlighted words,additional content data for the respective additional content identifiedin the search for each of the respective words identified in thescreening, and an identifier of the recipient (e.g., Betty's phone 14)to which the highlighted words were sent.

Thereafter, a subsequent recorded utterance from Betty's phone 14 may becommunicated 1221 to, and received and transcribed to text by, themobile communication service provider 140, whereupon the transcriptionis compared to the highlighted words that are maintained in associationwith the identifier for Betty's phone 14. Upon a match, it is determinedthat one of the highlighted words has been selected by Betty for displayof the additional content on Betty's phone 14. Consequently, theadditional content data maintained in association with the highlightedword of the match is sent to Betty's mobile phone 14 for display of theadditional content on Betty's mobile phone 14. The additional contentdata (C) may comprise the additional content for display or, preferably,the additional content data comprises an Internet web address. Theadditional content data is communicated 1223 to Betty's mobile phone 14,whereby, upon receipt, the mobile phone 14 is controlled to presentcontent to the user. In this regard, it will be appreciated that if thecontent data includes a web address (e.g., www.suzysushi.com), suchaddress may be passed to the web browser application of the mobile phone14 for display of the web page via the web browser.

It will be appreciated that in this implementation, the mobilecommunication service provider 140 may utilize the same technology asthe ATFS 18 in receiving, transcribing, and comparing an utterance tohighlighted words taken from messages for which words additional contenthas been identified. Moreover, the ability of the mobile communicationservice provider 140 to do this enables highlighting of words in textmessages from Betty to Adam even when Betty manually types the textmessages on her mobile phone 14.

For example, as shown in the message sequence of FIG. 10, Betty sends1225 a text message to Adam. The mobile communication service provider140 receives and screens the text message from Betty for appropriatewords for searching; for each appropriate word, the mobile communicationservice provider 140 searches for additional content; and those wordsfor which additional content is found in the search are thenhighlighted. The text message including the highlighted keywords—is thencommunicated 1227 to Adam's mobile phone 12. Additionally, the mobilecommunication service provider 140 maintains an association (preferablyin a database) of the highlighted words, additional content data for therespective additional content identified in the search for each of therespective words identified in the screening, and an identifier ofAdam's phone 12 to which the highlighted words were communicated 1227.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1229 and received and transcribed to text by the mobilecommunication service provider 140, whereupon the transcription iscompared to the highlighted words that are maintained in associationwith the identifier for Adam's phone 12. Upon a match, it is determinedthat one of the highlighted words has been selected by Adam for displayof the additional content on Adam's phone 12. Consequently, theadditional content data maintained in association with the highlightedword of the match is sent 1231 to Adam's mobile phone 12 for display ofthe additional content on Adam's mobile phone 12.

Fourth, fifth, and sixth implementations are illustrated, respectively,with reference to FIGS. 13-14, FIGS. 15-16, and FIG. 17. Thesimilarities with the first, second, and third implementations of FIGS.8-9, 10-11, and 12, respectively, will be appreciated. However, in thefourth, fifth, and sixth implementations, communications between Adamand Betty are effected by instant messaging (IM) using an IM client oneach mobile phone 12,14 in communication with an IM client on a serverof an IM service provider 240.

More specifically, in the fourth implementation, the ATFS 18 supportsvoice clicking, as in the first implementation. FIG. 13 is a graphicalillustration showing the sequence of messages in such an implementationof the invention, wherein content searching and keyword highlighting areperformed by the ATFS 18 in the portion of the communication system 10of FIG. 3; and FIG. 14 is a flowchart of steps in a method performed bythe ATFS 18 in accordance with the preferred implementation of FIG. 13.As shown in FIGS. 13 and 14, Adam utters a message to be transcribed andsent to Betty. The utterance is recorded in the form of audio data andis communicated 1301 to mobile communication service provider 140 andthen communicated 1303 to the ATFS 18. In accordance with thisimplementation, the ATFS 18 receives and transcribes 1401 the recordedutterance from Adam and screens 1403 the transcription for words thatare appropriate for highlighting.

For each word found in the screening 1403, the ATFS 18 searches 1405 foradditional content. Furthermore, the search preferably results in only“meaningful” additional content based on intelligent algorithms andrelevance scoring. Those words for which additional content is found inthe search are then highlighted 1407 by the ATFS 18, and the transcribedand filtered text, including the highlighted keywords, are communicated1305 to the mobile communication service provider 140 and thencommunicated 1307 to Adam's mobile phone 12. Additionally, the ATFS 18maintains 1411 an association of the highlighted words, additionalcontent data for the respective additional content identified in thesearch for each of the respective screened words, and an identifier ofthe recipient (e.g., Adam's phone 12) to which the highlighted wordswere sent 1409.

Thereafter, when a subsequent recorded utterance from Adam's phone 12 iscommunicated 1309,1311 and received 1413 by the ATFS 18, it is thentranscribed to text and compared 1415 to the highlighted words that aremaintained in association with the identifier for Adam's phone 12. Upona match, it is determined that one of the highlighted words has beenselected by Adam for display of the additional content on Adam's phone.Consequently, the additional content data maintained in association withthe highlighted word of the match is sent 1417 to Adam's mobile phone 12for display of the additional content on Adam's mobile phone 12. Theadditional content data (C) may comprise the additional content fordisplay or, preferably, the additional content data comprises anInternet web address. The additional content data is communicated1313,1315 to Adam's mobile phone 12, whereby, upon receipt, the mobilephone 12 is controlled to present content to the user. In this regard,it will be appreciated that if the content data includes a web address(e.g., www.suzysushi.com), such address may be passed to the web browserapplication of the mobile phone 12 for display of the web page via theweb browser.

As shown in the message sequence of FIG. 13, Adam next sends an instantmessage intended for Betty, which is communicated 1317,1319 to the IMservice provider 240. The instant message from Adam to Betty includesthe transcribed and filtered text received from the ATFS 18, but doesnot include the highlighted words from the ATFS 18. The instant messageis communicated 1321 to the mobile communication service provider 140,and then communicated 1323 to Betty's mobile phone 14. In response,Betty sends an instant message intended for Adam, which is communicated1325,1327 to the IM service provider 240. The instant message iscommunicated 1329 to the mobile communication service provider 140 andthen communicated 1323 to Adam's mobile phone 12.

In the fifth implementation, like the second implementation, the ATFS 18does not support voice clicking, which instead is supported by the IMservice provider 240. FIG. 15 is a graphical illustration showing thesequence of text messages in such a preferred implementation of theinvention, wherein content searching and keyword highlighting areperformed by the IM service provider 240 in the portion of thecommunication system 10 of FIG. 3; and FIG. 16 is a flowchart of stepsin a method performed by the IM service provider 240 in accordance withthe preferred implementation of FIG. 15.

Specifically, as illustrated in FIGS. 15 and 16, Adam first utters amessage to be sent to Betty. The utterance is recorded in the form ofaudio data and is communicated 1501 to mobile communication serviceprovider 140 and then communicated 1503 to the ATFS 18. The utterance istranscribed to text by the ATFS 18 and then communicated 1505,1507 backto Adam's phone 12. This part of the sequence is in accordance with thedisclosure of US Patent Application Publication No. 2007/0239837.

Adam next sends 1509 the transcribed text in an instant message. Inaccordance with this implementation, the mobile communication serviceprovider 140 communicates 1511 the instant message to the IM serviceprovider 240. The IM service provider 240 receives 1601 and screens 1603the instant message for appropriate words for searching. For eachappropriate word found in the screening 1603, the IM service provider240 searches 1605 for additional content.

Furthermore, the search preferably results in only “meaningful”additional content based on intelligent algorithms and relevancescoring. Those words for which additional content is found in the searchare then highlighted 1607 by the IM service provider 140. Preferably,the instant message—including the highlighted keywords—is thencommunicated 1513 to the mobile communication service provider 140 and,thereafter, is communicated 1515 to the recipient 1609. Additionally,the highlighted keywords also are communicated 1517 to the sender 1611of the instant message for updating of the log of the sent message withthe highlighted words for possible selection thereof by the sender ofthe instant message, and identifiers of both mobile phones 12,14preferably are maintained 1613 in association with the highlightedkeywords and respective additional content data. Alternatively, the IMservice provider 240 communicates the highlighted keywords only to therecipient, and only an identifier of the recipient's mobile phone ismaintained in the association.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1519,1521 and received 1615 and transcribed 1617 to text bythe IM service provider 240, whereupon the transcription is compared1617 to the highlighted words that are maintained in association withthe identifier for Adam's phone 12. Upon a match, it is determined thatone of the highlighted words has been selected for display of theadditional content. Consequently, the additional content data maintainedin association with the highlighted word of the match is sent 1619 toAdam's phone 12 for display of the additional content. The additionalcontent data is communicated 1523 to the mobile communication serviceprovider 140 and then is communicated 1525 to Adam's phone 12.

In addition, a subsequent recorded utterance from Betty's phone 14 maybe communicated 1527,1529 and received 1615 and transcribed 1617 to textby the IM service provider 240, whereupon the transcription is compared1617 to the highlighted words that are maintained in association withthe identifier for Betty's phone 14. Upon a match, it is determined thatone of the highlighted words has been selected for display of theadditional content. Consequently, the additional content data maintainedin association with the highlighted word of the match is sent 1619 toBetty's phone 14 for display of the additional content. The additionalcontent data is communicated 1531 to the mobile communication serviceprovider 140 and then is communicated 1533 to Betty's phone 14.

In each case, the additional content data (C) may comprise theadditional content for display or, preferably, the additional contentdata comprises an Internet web address. The additional content data iscommunicated to the respective mobile phone 12,14, whereby, uponreceipt, the mobile phone 12,14 is controlled to present content to theuser. In this regard, it will be appreciated that if the content dataincludes a web address (e.g., www.suzysushi.com), such address may bepassed to the web browser application of the mobile phone 12 for displayof the web page via the web browser.

It will be appreciated that in this implementation, the IM serviceprovider 240 may utilize the same technology as the ATFS 18 inreceiving, transcribing, and comparing an utterance to highlighted wordstaken from messages for which words additional content has beenidentified. Moreover, the ability of the IM service provider 240 to dothis enables highlighting of words in instant messages from Betty toAdam even when Betty manually types the text messages on her mobilephone 14.

For example, as shown in the message sequence of FIG. 15, Betty sends aninstant message to Adam. The instant message is communicated 1535,1537to the IM service provider 240. The IM service provider 240 receives andscreens the instant message from Betty for appropriate words forsearching; for each appropriate word, the IM service provider 240searches for additional content; and those words for which additionalcontent is found in the search are then highlighted. Preferably, theinstant message—including the highlighted keywords—is then communicated1539 to the mobile communication service provider 140 and, thereafter,both is communicated 1541 to the recipient (Adam), and communicated 1543to the sender (Betty) for updating of the log of the sent message onBetty's phone with the highlighted words. Identifiers of both mobilephones 12,14 preferably are maintained in association with thehighlighted keywords and respective additional content data.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1545,1547 and received and transcribed to text by the IMservice provider 240, whereupon the transcription is compared to thehighlighted words that are maintained in association with the identifierfor Adam's phone 12. Upon a match, it is determined that one of thehighlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Adam'sphone 12 for display of the additional content. The additional contentdata is communicated 1549 to the mobile communication service provider140 and then is communicated 1551 to Adam's phone 12.

In addition, a subsequent recorded utterance from Betty's phone 14 maybe communicated 1553,1555 and received and transcribed to text by the IMservice provider 240, whereupon the transcription is compared to thehighlighted words that are maintained in association with the identifierfor Betty's phone 14. Upon a match, it is determined that one of thehighlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Betty'sphone 14 for display of the additional content. The additional contentdata is communicated 1557 to the mobile communication service provider140 and then is communicated 1559 to Betty's phone 14.

In the sixth embodiment, voice clicking is supported separately andindependently by both the ATFS 18 and the IM service provider 240. FIG.17 is a graphical illustration showing the sequence of text messages insuch an additional preferred implementation of the invention, whereincontent searching and keyword highlighting are performed by both theATFS 18 and the IM service provider 240 in the portion of thecommunication system 10 of FIG. 3. With reference thereto, Adam utters amessage to be sent to Betty. The utterance is recorded in the form ofaudio data and is communicated 1701 to mobile communication serviceprovider 140 and then communicated 1703 to the ATFS 18. In accordancewith this implementation, the ATFS 18 receives and transcribes therecorded utterance from Adam and screens the transcription for wordsthat are appropriate for highlighting.

For each word found in the screening, the ATFS 18 searches foradditional content. Furthermore, the search preferably results in only“meaningful” additional content based on intelligent algorithms andrelevance scoring. Those words for which additional content is found inthe search are then highlighted by the ATFS 18, and the transcribed andfiltered text—including the highlighted keywords—are communicated 1705to the mobile communication service provider 140 and then communicated1707 to Adam's mobile phone 12. Additionally, the ATFS 18 maintains anassociation (preferably in a database) of the highlighted words,additional content data for the respective additional content identifiedin the search for each of the respective screened words, and anidentifier of the recipient (e.g., Adam's phone 12) to which thehighlighted words were sent.

Thereafter, when a subsequent recorded utterance from Adam's phone 12 iscommunicated 1709,1711 and received by the ATFS 18, it is thentranscribed to text and compared to the highlighted words that aremaintained in association with the identifier for Adam's phone 12. Upona match, it is determined that one of the highlighted words has beenselected by Adam for display of the additional content on Adam's phone.Consequently, the additional content data maintained in association withthe highlighted word of the match is sent to Adam's mobile phone 12 fordisplay of the additional content on Adam's mobile phone 12. Theadditional content data (C) may comprise the additional content fordisplay or, preferably, the additional content data comprises anInternet web address. The additional content data is communicated1713,1715 to Adam's mobile phone 12, whereby, upon receipt, the mobilephone 12 is controlled to present content to the user. In this regard,it will be appreciated that if the content data includes a web address(e.g., www.suzysushi.com), such address may be passed to the web browserapplication of the mobile phone 12 for display of the web page via theweb browser.

As shown in the message sequence of FIG. 17, Adam then sends 1717,1719an instant message to Betty. The instant message from Adam to Bettyincludes the transcribed and filtered text received from the ATFS 18,but does not include the highlighted words from the ATFS 18. Inaccordance with this implementation, the IM service provider 240receives and screens the instant message for appropriate words forsearching. For each appropriate word found in the screening, the IMservice provider 240 searches for additional content.

Furthermore, the search preferably results in only “meaningful”additional content based on intelligent algorithms and relevancescoring. Those words for which additional content is found in the searchare then highlighted by the IM service provider 140. Preferably, theinstant message—including the highlighted keywords—is then communicated1721 to the mobile communication service provider 140 and, thereafter,both is communicated 1723 to the recipient of the instant message, andcommunicated 1725 to the sender of the instant message for updating ofthe sent message on the sender's mobile phone with highlighting.Identifiers of both mobile phones 12,14 preferably are maintained inassociation with the highlighted keywords and respective additionalcontent data. Alternatively, the IM service provider 240 communicatesthe highlighted keywords only to the recipient of the instant message,and only the mobile phone of the recipient is maintained in theassociation.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1727,1729 and received and transcribed to text by the IMservice provider 240, whereupon the transcription is compared to thehighlighted words that are maintained in association with the identifierfor Adam's phone 12. Upon a match, it is determined that one of thehighlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Adam'sphone 12 for display of the additional content. The additional contentdata is communicated 1731 to the mobile communication service provider140 and then is communicated 1733 to Adam's phone 12.

In addition, a subsequent recorded utterance from Betty's phone 14 maybe communicated 1735,1737 to, and received and transcribed to text by,the IM service provider 240, whereupon the transcription is compared tothe highlighted words that are maintained in association with theidentifier for Betty's phone 14. Upon a match, it is determined that oneof the highlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Betty'sphone 14 for display of the additional content. The additional contentdata is communicated 1739 to the mobile communication service provider140 and then is communicated 1741 to Betty's phone 14.

In each case, the additional content data (C) may comprise theadditional content for display or, preferably, the additional contentdata comprises an Internet web address. The additional content data iscommunicated to the respective mobile phone 12,14, whereby, uponreceipt, the mobile phone 12,14 is controlled to present content to theuser. In this regard, it will be appreciated that if the content dataincludes a web address (e.g., www.suzysushi.com), such address may bepassed to the web browser application of the mobile phone 12 for displayof the web page via the web browser.

It will be appreciated that in this implementation, the IM serviceprovider 240 may utilize the same technology as the ATFS 18 inreceiving, transcribing, and comparing an utterance to highlighted wordstaken from messages for which words additional content has beenidentified. Moreover, the ability of the IM service provider 240 to dothis enables highlighting of words in instant messages from Betty toAdam even when Betty manually types the text messages on her mobilephone 14.

For example, as shown in the message sequence of FIG. 17, Betty sends aninstant message to Adam. The instant message is communicated 1743,1745to the IM service provider 240. The IM service provider 240 receives andscreens the instant message from Betty for appropriate words forsearching; for each appropriate word, the IM service provider 240searches for additional content; and those words for which additionalcontent is found in the search are then highlighted. Preferably, theinstant message—including the highlighted keywords—is then communicated1747 to the mobile communication service provider 140 and, thereafter,both is communicated 1749 to the receiver, and communicated 1751 to thesender, for updating of the sent message with highlighting. Identifiersof both mobile phones 12,14 preferably are maintained in associationwith the highlighted keywords and respective additional content data.Alternatively, the IM service provider 240 communicates the highlightedkeywords only to the receiver.

Thereafter, a subsequent recorded utterance from Adam's phone 12 may becommunicated 1753,1755 and received and transcribed to text by the IMservice provider 240, whereupon the transcription is compared to thehighlighted words that are maintained in association with the identifierfor Adam's phone 12. Upon a match, it is determined that one of thehighlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Adam'sphone 12 for display of the additional content. The additional contentdata is communicated 1757 to the mobile communication service provider140 and then is communicated 1759 to Adam's phone 12.

In addition, a subsequent recorded utterance from Betty's phone 14 maybe communicated 1761,1763 and received and transcribed to text by the IMservice provider 240, whereupon the transcription is compared to thehighlighted words that are maintained in association with the identifierfor Betty's phone 14. Upon a match, it is determined that one of thehighlighted words has been selected for display of the additionalcontent. Consequently, the additional content data maintained inassociation with the highlighted word of the match is sent to Betty'sphone 14 for display of the additional content. The additional contentdata is communicated 1765 to the mobile communication service provider140 and then is communicated 1767 to Betty's phone 14.

It should be particularly noted that, as described above, in the fifthand sixth implementations, wherein the IM service provider 240 supportsvoice clicking, both Adam and Betty preferably are able to selectkeywords that are highlighted in a message, regardless of whether Adamor Betty is the recipient of the message. In this respect, the IMservice provider 240 preferably communicates 1515,1517 and 1541,1543 and1723,1725 and 1749,1751 the highlighted keywords to both sender andreceiver for a particular message, and identifiers of both mobile phones12,14 preferably are maintained in association with the highlightedkeywords and respective additional content data. In FIGS. 15 and 17,examples of both Adam and Betty performing voice clicks are shown.Alternatively, the IM service provider 240 communicates the highlightedkeywords to the receiver.

As will be apparent from consideration of these foregoing siximplementations, as text messages are propagated through a communicationsystem 10, one or more of the elements of the system 10 may be equippedto conduct the screening, content searching, and keyword highlighting inaccordance with the invention. Furthermore, the additional content couldbe advertising content, location data, general search information,reference information (sponsored or unsponsored) or the like, and couldinclude video, audio, or audiovisual content.

Moreover, the screening may be configured to identify not onlyconventional words, but also text speech, including words like “2nite”,which would be equated to “tonight” and handled in the same manner.

Furthermore, the teachings of the present invention are not confined tothe English language, but are equally applicable in many other languagesas well.

Optionally, in the event that there are multiple possibleinterpretations of an utterance in comparing the transcription thereofto maintained keywords, additional content data may be returned forevery possible/known interpretation (thereby resulting, for example, inmultiple tabs opening in a web browser), or only the additional contentdata corresponding to the highest confidence in the match may bereturned.

Still yet, it is contemplated that that highlighted keywords will bedisplayed, but it will not be possible to speak them in order to expandthe additional content for viewing. This may be because the user hasentered a noisy environment, or has entered a very quiet environmentwhere it would be disruptive to speak. In this case, a manual method ofselecting highlighted keywords is made available to augmentvoice-clicking. This may be keyboard navigation, navigation via a touchpanel, navigation via a pointing device, or any other conventionalnavigation means.

It also will be appreciated that the association maintained between thekeywords, the respective additional content data, and the identifiersmay be kept for a short period of time (minutes or hours), or a longerperiod of time (days, weeks, or months). Accordingly, a user mayretrieve the additional content in certain situations if that userremembers the keyword.

Furthermore, voice commands or cues may be used to indicate that anutterance is intended to be a keyword and that the additional contentdata is desired to be sent. In this regard, the cue may be “Go” andused, for example, by sending recorded audio representing the utterance“Go Sushi”. Additionally, it is contemplated that selection of anappropriate button on the user's mobile device will indicate the desiredrecipient of an utterance, and utterances in certain scenarios may beappropriate for sending to more than one recipient (such as in the thirdimplementation, wherein both the ATFS 18 and mobile communicationservice provider 140 transcribe utterances, or in the sixthimplementation, wherein both the ATFS 18 and IM service provider 240transcribe utterances).

It will also be appreciated that in some embodiments of the invention,the transcription of one or more utterances can be performed by a mobilecommunication device rather than by, for example, a backend server. Forinstance, in some embodiments an utterance in performance of a voiceclick may be transcribed by the mobile communication device rather thanby a server.

Based on the foregoing description, it will be readily understood bythose persons skilled in the art that the present invention issusceptible of broad utility and application. Many embodiments andadaptations of the present invention other than those specificallydescribed herein, as well as many variations, modifications, andequivalent arrangements, will be apparent from or reasonably suggestedby the present invention and the foregoing descriptions thereof, withoutdeparting from the substance or scope of the present invention.

Accordingly, while the present invention has been described herein indetail in relation to one or more preferred embodiments, it is to beunderstood that this disclosure is only illustrative and exemplary ofthe present invention and is made merely for the purpose of providing afull and enabling disclosure of the invention. The foregoing disclosureis not intended to be construed to limit the present invention orotherwise exclude any such other embodiments, adaptations, variations,modifications or equivalent arrangements, the present invention beinglimited only by the claims appended hereto and the equivalents thereof.

For example, in one alternative embodiment of the invention in which atranscript is played on the mobile communication device for verificationbefore sending, instead of aural cueing, tactile cuing is used. Forexample, when a word is played back for which additional content isavailable, the mobile phone may vibrate in order to emphasize such word.Such tactile cueing may be used in place of, or in addition to, auralcueing.

Additionally, rather than present additional content upon utterance ofan emphasized word in accordance with the foregoing embodiments andimplementations of the invention, some other action may occur inaddition to, or in substitution for, the presentation of such additionalcontent, such as, for example, the opening of an application on themobile communication device and the calling of a telephone number.

1. A computer-implemented method of facilitating presentation ofadditional content for a word that is part of a transcribed utterance,the computer-implemented method comprising: under control of one or morecomputing devices configured with specific computer executableinstructions, (a) first, (i) receiving first audio data communicatedfrom a mobile communication device; (ii) transcribing the first audiodata to first text; (iii) screening the first text to identify one ormore words that are appropriate for searching; (iv) searching foradditional content in one or more databases based on the one or morewords identified as being appropriate for searching; (v) selecting aword for which respective additional content is found in the searching;(vi) communicating the first text to the mobile communication devicealong with information identifying the selected word for whichrespective additional content is found in the searching; and (vii)maintaining in a database an association between the selected word, therespective additional content for the selected word, and an identifierof the mobile communication device; (b) thereafter, (i) receiving secondaudio data communicated from the mobile communication device and theidentifier of the mobile communication device, the second audio datarepresenting an utterance that includes the selected word; (ii)transcribing the second audio data to second text; (iii) determiningthat the identifier of the mobile communication device is maintained inthe database and, based at least in part upon the identifier of themobile communication device being maintained in the database, comparingthe second text to the selected word maintained in the database inassociation with the identifier of the mobile communication device; and(iv) based at least in part on the selected word being present in thesecond text, communicating to the mobile communication device therespective additional content associated with the selected word, whereinthe communicated respective additional content is for presentation bythe mobile communication device.
 2. The computer-implemented method ofclaim 1, wherein communicating the first text to the mobilecommunication device along with information identifying the selectedword for which respective additional content is found in the searchingcomprises highlighting the selected word such that the selected word isemphasized when displayed on the mobile communication device.
 3. Thecomputer-implemented method of claim 1, wherein communicating the firsttext to the mobile communication device along with informationidentifying the selected word for which respective additional content isfound in the searching comprises configuring an audio presentation ofthe first text to include pronunciation emphasis of the selected word inthe first text for which respective additional content is found in thesearching.
 4. The computer-implemented method of claim 1, whereincommunicating the first text to the mobile communication device alongwith information identifying the selected word for which respectiveadditional content is found in the searching comprises configuring anaudio presentation of the first text to include aural cueing for theselected word in the first text for which respective additional contentis found in the searching.
 5. The computer-implemented method of claim1, wherein the first audio data that is received from the mobilecommunication device includes an indication that the first audio data isintended to be at least a portion of the text of a message.
 6. Thecomputer-implemented method of claim 5, wherein the indication comprisesthat the message is an instant message.
 7. The computer-implementedmethod of claim 5, wherein the indication comprises that the message isto be communicated in accordance with a text messaging protocol.
 8. Thecomputer-implemented method of claim 5, further comprising filtering thefirst text that is transcribed from the first audio data based on theindication.
 9. A non-transitory computer-readable medium whose contentsconfigure one or more computing systems to perform a method offacilitating presentation of additional content for a word that is partof a transcribed utterance intended for a message, the methodcomprising: (a) first, (i) receiving first audio data communicated froma mobile communication device; (ii) transcribing the first audio data tofirst text; (iii) screening the first text to identify one or more wordsthat are appropriate for searching; (iv) searching for additionalcontent in one or more databases based on the one or more wordsidentified as being appropriate for searching; (v) selecting a word forwhich respective additional content is found in the searching; (vi)communicating the first text to the mobile communication device alongwith information identifying the selected word for which respectiveadditional content is found in the searching; and (vii) maintaining in adatabase an association between the selected word, the respectiveadditional content for the selected word, and an identifier of themobile communication device; (b) thereafter, (i) receiving second audiodata communicated from the mobile communication device and theidentifier of the mobile communication device, the second audio datarepresenting an utterance that includes the selected word; (ii)transcribing the second audio data to second text; (iii) determiningthat the identifier of the mobile communication device is maintained inthe database and, based at least in part upon the identifier of themobile communication device being maintained in the database, comparingthe second text to the selected word maintained in the database inassociation with the identifier of the mobile communication device; and(iv) based at least in part on the selected word being present in thesecond text, communicating to the mobile communication device therespective additional content associated with the selected word, whereinthe communicated respective additional content is for presentation bythe mobile communication device.
 10. The non-transitory computerreadable medium of claim 9, wherein the searching for additional contentincludes searching based on the selected word using an Internet searchengine and analyzing the results provided in response thereto.
 11. Thenon-transitory computer readable medium of claim 9, wherein therespective additional content comprises a web address for passing to aweb browser of the mobile communication device.
 12. The non-transitorycomputer readable medium of claim 9, wherein the respective additionalcontent comprises a URL for presentation by the mobile communicationdevice.
 13. The non-transitory computer readable medium of claim 9,wherein the searching includes sending a keyword to a third party SMSservice and analyzing the results provided in response thereto.
 14. Thenon-transitory computer readable medium of claim 9, wherein searchingfor additional content comprises communicating a keyword to a thirdparty SMS service, in response to which the respective additionalcontent found in the searching is provided.
 15. The non-transitorycomputer readable medium of claim 9, wherein the utterance furtherincludes a cue that the selected word is included in the utterance forcommunicating the respective additional content for the selected word.16. The non-transitory computer readable medium of claim 9, wherein theassociation between the selected word, the respective additional contentfor the selected word, and the identifier of the mobile communicationdevice is maintained in the database for an extended period of timecomprising a plurality of days or weeks, whereby additional content maybe provided for the selected word communicated to the mobilecommunication device even though a plurality of days have passed sincethe selected word was communicated to the mobile communication device.17. A system for facilitating presentation of additional content for aword that is part of a transcribed utterance, the system comprising oneor more processors configured to: (a) first, (i) receive first audiodata communicated from a mobile communication device; (ii) transcribethe first audio data to first text; (iii) screen the first text toidentify one or more words that are appropriate for searching; (iv)search for additional content in one or more databases based on the oneor more words identified as being appropriate for searching; (v) selecta word for which respective additional content is found in thesearching; (vi) communicate the first text to the mobile communicationdevice along with information identifying the selected word for whichrespective additional content is found in the searching; and (vii)maintain in a database an association between the selected word, therespective additional content for the selected word, and an identifierof the mobile communication device; (b) thereafter, (i) receive secondaudio data communicated from the mobile communication device and theidentifier of the mobile communication device, the second audio datarepresenting an utterance that includes the selected word; (ii)transcribe the second audio data to second text; (iii) determine thatthe identifier of the mobile communication device is maintained in thedatabase and, based at least in part upon the identifier of the mobilecommunication device being maintained in the database, compare thesecond text to the selected word maintained in the database inassociation with the identifier of the mobile communication device; and(iv) based at least in part on the selected word being present in thesecond text, communicate the respective additional content associatedwith the selected word to the mobile communication device, wherein thecommunicated respective additional content is for presentation by themobile communication device.
 18. The system of claim 17, wherein therespective additional content comprises a web address.
 19. The system ofclaim 17, wherein the respective additional content comprises a responsereceived after sending a keyword to a third party SMS service.